Web Survey Bibliography
Relevance & Research Question: Attrition is an important methodological challenge to panel surveys (Lynn 2009). Still, there is a remarkable shortage of variables which are associated with both, the propensity of respondents to stay in the panel and the variables of interest. As a result, propensity score weights which are designed to correct for this type of nonresponse frequently yield mixed results.
This paper addresses the question whether paradata can successfully be applied to improve the prediction of attrition in panel Web surveys. Their main advantage is that they are collected as a byproduct of the survey process. However, it is still an open question which paradata can be used to model attrition and to what extent these paradata are correlated with variables of interest (Kreuter and Olson 2013).
Methods & Data: We use logistic regressions to model attrition in a 7-wave panel Web survey and to compute propensity score weights. The models are fitted with sets of socio-demographic, substantial, survey evaluation, and paradata variables. The latter include measures of response times, user agent strings to determine the device used by the respondent, as well as indicators of the respondents’ response behavior. Finally, we use supplemental cross-sectional Web surveys to assess the effectiveness of propensity score weights based on different sets of variables.
Results: Our results show that including paradata significantly improves the prediction of panel attrition. However, the paradata variables do not supersede socio-demographic, survey evaluation and substantial variables, but they complement them. Yet, the paradata are only moderately correlated with variables of interest at best. As a result, including paradata does not significantly improve the effectiveness of propensity score weights.
Added Value: This paper enhances the existing knowledge in several ways: It presents a set of paradata variables and provides empirical tests of their capability to explain attrition. We show that these paradata can successfully be used to create auxiliary data in a cost-efficient way. At the same time, we demonstrate that they do not ultimately help to correct for panel attrition. Thus, we conclude that further research on paradata, panel attrition and its correction is needed.
Conference Homepage (abstract) / (presentation) >>
Web survey bibliography (183)
- Using experts’ consensus (the Delphi method) to evaluate weighting techniques in web surveys not...; 2017; Toepoel, V.; Emerson, H.
- A Partially Successful Attempt to Integrate a Web-Recruited Cohort into an Address-Based Sample; 2017; Kott, P. S., Farrelly, M., Kamyab, K.
- Overview: Online Surveys; 2017; Vehovar, V.; Lozar Manfreda, K.
- Inferences from Internet Panel Studies and Comparisons with Probability Samples; 2016; Lachan, R.; Boyle, J.; Harding, R.
- Integration of a phone-based household travel survey and a web-based student travel survey; 2016; Verreault, H.; Morency, C.
- Estimation and Adjustment of Self-Selection Bias in Volunteer Panel Web Surveys ; 2016; Niu, Ch.
- Calculating Standard Errors for Nonprobability Samples when Matching to Probability Samples ; 2016; Lee, Ad.; ZuWallack, R. S.
- Establishing the accuracy of online panels for survey research; 2016; Bruggen, E.; van den Brakel, J.; Krosnick, J. A.
- Evaluating Three Approaches to Statistically Adjust for Mode Effects; 2016; Kolenikov, S.; Kennedy, C.
- Linearization Variance Estimators for Mixed ‒ mode Survey Data when Response Indicators are Modeled...; 2016; Demnati, A.
- Options for Fielding and Analyzing Web Surveys; 2016; Schonlau, M.; Couper, M. P.
- Report of the Inquiry into the 2015 British general election opinion polls; 2016; Sturgis, P., Baker, N., Callegaro, M., Fisher, St., Green, J., Jennings, W., Kuha, J., Lauderdale, B...
- Solving the Nonresponse Problem With Sample Matching?; 2016
- Online and Social Media Data As an Imperfect Continuous Panel Survey; 2016; Diaz, F.; Garmon, F.; Hofman, J. K.; Kiciman, E.; Rothschild, D.
- Quota Controls in Survey Research.; 2016; Gittelman, S. H.; Thomas, R. K.; Lavrakas, P. J.; Lange, V.
- Scientific Surveys Based on Incomplete Sampling Frames and High Rates of Nonresponse; 2016; Fahimi, M.; Barlas, F. M.; Thomas, R. K.; Buttermore, N. R.
- Doing Surveys Online ; 2016; Toepoel, V.
- Using Mobile Phones for High-Frequency Data Collection; 2015; Azevedo, J. P.; Ballivian, A.; Durbin, W.
- On Bias Adjustments for Web Surveys; 2015; Fan, L.; Lou, W.; Landsman, V.
- The quality of data collected using online panels: a decade of research ; 2015; Callegaro, M.
- Does the use of mobile devices (tablets and smartphones) affect survey quality and choice behaviour...; 2015; Liebe, U., Glenk, K., Oehlmann, M., Meyerhoff, J.
- Web-based survey, calibration, and economic impact assessment of spending in nature based recreation; 2015; Paudel, K. P., Devkota, N., Gyawali, B.
- Using Web Panels for Official Statistics; 2014; Bethlehem, J.
- Self-reported cheating in web surveys on political knowledge; 2014; Jensen, C., Thomsen, J. P. F.
- Keeping Surveys Valid, Reliable, and Useful: A Tutorial; 2014; Greenberg, M. R., Weiner, M. D.
- Prioritisation of alternatives with analytical hierarchy process plus response latency and web survey...; 2014; Barone, S. Errore, A., Lombardo, A.
- A critical review of studies investigating the quality of data obtained with online panels based on...; 2014; Callegaro, M., Villar, A., Yeager, D. S., Krosnick, J. A.
- Online panel research: History, concepts, applications and a look at the future; 2014; Callegaro, M., Baker, R., Bethlehem, J., Goeritz, A., Krosnick, J. A., Lavrakas, P. J.
- Using Paradata to Predict and to Correct for Panel Attrition in a Web-based Panel Survey; 2014; Rossmann, J., Gummer, T.
- Improving cheater detection in web-based randomized response using client-side paradata; 2014; Dombrowski, K., Becker, C.
- Modelling ”don’t know” responses in rating scales; 2014; Manisera, M., Zuccolotto, P.
- User Modeling via Machine Learning and Rule-Based Reasoning to Understand and Predict Errors in Survey...; 2013; Stuart, L. C.
- Comparison of Three Modes for a Crime Victimization Survey; 2013; Laaksonen, S., Heiskanen, M.
- The Short-term Campaign Panel of the German Longitudinal Election Study 2009. Design, Implementation...; 2013; Steinbrecher, M., Rossmann, J.
- Too Fast, Too Straight, Too Weird: Post Hoc Identification of Meaningless Data in Internet ; 2013; Leiner, D. J.
- Assessing Nonresponse Bias in the Green Technologies and Practices Survey; 2013; Meekins, B., Sverchkov, M., Stang, S.
- Web Panel Representativeness; 2013; Bianchi, A., Biffignandi, S.
- On the Impact of Response Patterns on Survey Estimates from Access Panels; 2013; Enderle, T., Muennich, R., Bruch, C.
- Unit Nonresponse and Weighting Adjustments: A Critical Review; 2013; Brick, J. M.
- Adjusting for bias in a mixed-mode CAWI survey on University students ; 2013; Clerici, R., Giraldo, A.
- A probability-based web panel for the UK: What could it look like?; 2013; Nicolaas, G.
- Panel Attrition: Separating Stayers, Sleepers and Other Types of Drop-Out in an Internet Panel; 2013; Lugtig, P. J.
- Speeding and Non-Differentiation in Web Surveys: Evidence of Correlation and Strategies for Reduction...; 2013; Zhang, Che.
- Web Versus Outbound: A Mode Face-Off Following the Presidential Debate; 2013; Marlar, J.
- The Effects of Errors in Paradata on Weighting Class Adjustments: A Simulation Study; 2013; West, B. T.
- Practical tools for designing and weighting survey samples; 2013; Valliant, R. L., Daver, J. A., Kreuter, F.
- Moving an established survey online – or not?; 2013; Barber, T., Chilvers, D., Kaul, S.
- Measuring working conditions in a volunteer web survey; 2013; de Pedraza, P., Villacampa, A.
- Propensity Score Weighting – Can Personality Adjust for Selectivity?; 2013; Glantz, A., Greszki, R.
- Eurobarometer Special surveys: Special Eurobarometer 381; 2012